Method and system for dynamic traffic distribution and bi-casting in a hybrid network environment

ABSTRACT

A method, apparatus and computer program product are provided in accordance with example embodiments in order to provide for the efficient, dynamic distribution of traffic in a hybrid network environment based at least in part on reliability probabilities associated with individual subflows within the network. In some example implementations, a traffic distribution entity provides for control over the determination of combined reliability probabilities of multiple potential traffic distribution modes and the selection of a traffic distribution mode that is capable of meeting performance targets, such as those associated with mission-critical operations of cyber-physical systems.

TECHNICAL FIELD

An example embodiment relates generally to communications networktechnology, particularly in context of hybrid networks that combineloosely coupled technologies in order to provide support formission-critical device operations that require high communicationreliability within particularized timing constraints.

BACKGROUND

As modern communications networks have become increasingly ubiquitousand powerful, an ever-increasing number of devices and types of devicesthat are capable of interacting with such communications networks havebeen introduced. Advancements in communications network technology andin the devices that are capable of interacting with one or more networkshave trended towards the development and introduction of cyber-physicalsystems (“CPS”) and other devices and systems that rely on networkconnectivity to support mission-critical operations and functions.

Particularly in situations where a cyber-physical system or anothersystem is likely to operate in a physical environment near individuals,objects, and/or structures, it is often essential that the connectionbetween such a system and a network should be highly reliable and ensurethat communications are sent and received by the system withinparticularized timing constraints. The technical challenges associatedwith meeting the requirements of such mission-critical operations areoften compounded in situations involving cyber-physical systems and/orother system based on mobile and/or other wireless systems whichpotentially encounter multiple modern and/or legacy networks. Theinventors of the invention disclosed herein have identified these andother technical challenges, and developed the solutions described andotherwise referenced herein.

BRIEF SUMMARY

A method, apparatus and computer program product are therefore providedin accordance with an example embodiment in order to provide methods,apparatuses, and/or systems that provide for efficient, dynamicdistribution of traffic in a hybrid network environment based at leastin part on reliability probabilities associated with individual subflowswithin the network.

In an example embodiment, a method is provided that includes receiving,at a traffic distribution entity in a hybrid network environment, afirst reliability probability indicator associated with a first accessnetwork and a second reliability probability indicator associated with asecond access network; determining, based at least in part on the firstreliability probability indicator and the second reliability probabilityindicator, a combined reliability probability for each trafficdistribution mode within a plurality of traffic distribution modes;ranking each traffic distribution mode with a combined reliabilityprobability above a target reliability probability; and selecting atraffic distribution mode with a combined reliability probability abovethe target reliability probability.

In some example implementations of such a method, the first reliabilityprobability indicator is an indicator of a composite metric of a packetloss ratio and a latency associated with the first access network andthe second reliability probability indicator is an indicator of acomposite metric of a packet loss ratio and a latency associated withthe second access network. In some such example implementations, and inother example implementations, the hybrid network environment is amulti-path transmission control protocol (MPTCP) environment.

In some such example implementations, and in other exampleimplementations, the first reliability indicator and the secondreliability indicator are associated with at least part of acyberphysical system or a high-reliability low-latency communicationsystem operating within the hybrid network; the combined reliabilityprobability for each traffic distribution mode within the plurality oftraffic distribution modes is associated with at least part of thecyberphysical system or the high-reliability low-latency communicationsystem operating within the hybrid network; and the target reliabilityprobability comprises a real-time reliability requirement associatedwith the cyberphysical system or the high-reliability low-latencycommunication system operating within the hybrid network. In some suchexample implementations, and in other example implementations, rankingeach traffic distribution mode with a combined reliability probabilityabove a target reliability probability comprises applying a predefinedcost function to each traffic distribution mode.

In some such example implementations, and in other exampleimplementations, selecting the traffic distribution mode with a combinedreliability probability above the target reliability probabilitycomprises causing a transmission to be directed to the cyberphysicalsystem via one subflow associated with either the first access networkor the second access network. In some such example implementations, andin other example implementations, selecting the traffic distributionmode with a combined reliability probability above the targetreliability probability comprises causing a bicasted transmission to bedirected to the cyberphysical system via one subflow associated with thefirst access network and via one subflow associated with the secondaccess network. In some such example implementations, and in otherexample implementations, selecting the traffic distribution mode with acombined reliability probability above the target reliabilityprobability comprises causing a transmission to be directed to thecyberphysical system via one subflow associated with the first accessnetwork and causing a bicasted version of the transmission to bedirected to the cyberphysical system via the second access network. Insome such example implementations, and in other example implementations,selecting a traffic distribution mode with a combined reliabilityprobability above the target reliability probability comprises selectingthe traffic distribution mode with the highest ranking.

In another example embodiment, an apparatus is provided that includes atleast one processor and at least one memory that includes computerprogram code with the at least one memory and the computer program codeconfigured to, with the at least one processor, cause the apparatus toat least receive, in a hybrid network environment, a first reliabilityprobability indicator associated with a first access network and asecond reliability probability indicator associated with a second accessnetwork; determine, based at least in part on the first reliabilityprobability indicator and the second reliability probability indicator,a combined reliability probability for each traffic distribution modewithin a plurality of traffic distribution modes; rank each trafficdistribution mode with a combined reliability probability above a targetreliability probability; and select a traffic distribution mode with acombined reliability probability above the target reliabilityprobability.

In some example implementations of such an apparatus, the firstreliability probability indicator is an indicator of a composite metricof a packet loss ratio and a latency associated with the first accessnetwork and the second reliability probability indicator is an indicatorof a composite metric of a packet loss ratio and a latency associatedwith the second access network. In some such example implementations,and in other example implementations, the hybrid network environment isa multi-path transmission control protocol (MPTCP) environment.

In some such example implementations, and in other exampleimplementations, the first reliability indicator and the secondreliability indicator are associated with at least part of acyberphysical system or a high-reliability low-latency communicationsystem operating within the hybrid network; the combined reliabilityprobability for each traffic distribution mode within the plurality oftraffic distribution modes is associated with at least part of thecyberphysical system or the high-reliability low-latency communicationsystem operating within the hybrid network; and the target reliabilityprobability comprises a real-time reliability requirement associatedwith the cyberphysical system or the high-reliability low-latencycommunication system operating within the hybrid network. In some suchexample implementations, and in other example implementations, rankingeach traffic distribution mode with a combined reliability probabilityabove a target reliability probability comprises applying a predefinedcost function to each traffic distribution mode.

In some such example implementations, and in other exampleimplementations, selecting the traffic distribution mode with a combinedreliability probability above the target reliability probabilitycomprises causing a transmission to be directed to the cyberphysicalsystem via one subflow associated with either the first access networkor the second access network. In some such example implementations, andin other example implementations, selecting the traffic distributionmode with a combined reliability probability above the targetreliability probability comprises causing a bicasted transmission to bedirected to the cyberphysical system via one subflow associated with thefirst access network and via one subflow associated with the secondaccess network. In some such example implementations, and in otherexample implementations, selecting the traffic distribution mode with acombined reliability probability above the target reliabilityprobability comprises causing a transmission to be directed to thecyberphysical system via one subflow associated with the first accessnetwork and causing a bicasted version of the transmission to bedirected to the cyberphysical system via the second access network. Insome such example implementations, and in other example implementations,selecting a traffic distribution mode with a combined reliabilityprobability above the target reliability probability comprises selectingthe traffic distribution mode with the highest ranking.

In a further example embodiment, a computer program product is providedthat includes at least one non-transitory computer-readable storagemedium having computer-executable program code instructions storedtherein with the computer-executable program code instructions includingprogram code instructions configured to receive, in a hybrid networkenvironment, a first reliability probability indicator associated with afirst access network and a second reliability probability indicatorassociated with a second access network; determine, based at least inpart on the first reliability probability indicator and the secondreliability probability indicator, a combined reliability probabilityfor each traffic distribution mode within a plurality of trafficdistribution modes; rank each traffic distribution mode with a combinedreliability probability above a target reliability probability; andselect a traffic distribution mode with a combined reliabilityprobability above the target reliability probability.

In some example implementations of such computer program product, thefirst reliability probability indicator is an indicator of a compositemetric of a packet loss ratio and a latency associated with the firstaccess network and the second reliability probability indicator is anindicator of a composite metric of a packet loss ratio and a latencyassociated with the second access network. In some such exampleimplementations, and in other example implementations, the hybridnetwork environment is a multi-path transmission control protocol(MPTCP) environment.

In some such example implementations, and in other exampleimplementations, the first reliability indicator and the secondreliability indicator are associated with at least part of acyberphysical system or a high-reliability low-latency communicationsystem operating within the hybrid network; the combined reliabilityprobability for each traffic distribution mode within the plurality oftraffic distribution modes is associated with at least part of thecyberphysical system or the high-reliability low-latency communicationsystem operating within the hybrid network; and the target reliabilityprobability comprises a real-time reliability requirement associatedwith the cyberphysical system or the high-reliability low-latencycommunication system operating within the hybrid network. In some suchexample implementations, and in other example implementations, rankingeach traffic distribution mode with a combined reliability probabilityabove a target reliability probability comprises applying a predefinedcost function to each traffic distribution mode.

In some such example implementations, and in other exampleimplementations, selecting the traffic distribution mode with a combinedreliability probability above the target reliability probabilitycomprises causing a transmission to be directed to the cyberphysicalsystem via one subflow associated with either the first access networkor the second access network. In some such example implementations, andin other example implementations, selecting the traffic distributionmode with a combined reliability probability above the targetreliability probability comprises causing a bicasted transmission to bedirected to the cyberphysical system via one subflow associated with thefirst access network and via one subflow associated with the secondaccess network. In some such example implementations, and in otherexample implementations, selecting the traffic distribution mode with acombined reliability probability above the target reliabilityprobability comprises causing a transmission to be directed to thecyberphysical system via one subflow associated with the first accessnetwork and causing a bicasted version of the transmission to bedirected to the cyberphysical system via the second access network. Insome such example implementations, and in other example implementations,selecting a traffic distribution mode with a combined reliabilityprobability above the target reliability probability comprises selectingthe traffic distribution mode with the highest ranking.

In yet another example embodiment, an apparatus is provided thatincludes means for receiving, at a traffic distribution entity in ahybrid network environment, a first reliability probability indicatorassociated with a first access network and a second reliabilityprobability indicator associated with a second access network;determining, based at least in part on the first reliability probabilityindicator and the second reliability probability indicator, a combinedreliability probability for each traffic distribution mode within aplurality of traffic distribution modes; ranking each trafficdistribution mode with a combined reliability probability above a targetreliability probability; and selecting a traffic distribution mode witha combined reliability probability above the target reliabilityprobability.

In some example implementations of such an apparatus, the firstreliability probability indicator is an indicator of a composite metricof a packet loss ratio and a latency associated with the first accessnetwork and the second reliability probability indicator is an indicatorof a composite metric of a packet loss ratio and a latency associatedwith the second access network. In some such example implementations,and in other example implementations, the hybrid network environment isa multi-path transmission control protocol (MPTCP) environment.

In some such example implementations, and in other exampleimplementations, the first reliability indicator and the secondreliability indicator are associated with at least part of acyberphysical system or a high-reliability low-latency communicationsystem operating within the hybrid network; the combined reliabilityprobability for each traffic distribution mode within the plurality oftraffic distribution modes is associated with at least part of thecyberphysical system or the high-reliability low-latency communicationsystem operating within the hybrid network; and the target reliabilityprobability comprises a real-time reliability requirement associatedwith the cyberphysical system or the high-reliability low-latencycommunication system operating within the hybrid network. In some suchexample implementations, and in other example implementations, rankingeach traffic distribution mode with a combined reliability probabilityabove a target reliability probability comprises applying a predefinedcost function to each traffic distribution mode.

In some such example implementations, and in other exampleimplementations, selecting the traffic distribution mode with a combinedreliability probability above the target reliability probabilitycomprises causing a transmission to be directed to the cyberphysicalsystem via one subflow associated with either the first access networkor the second access network. In some such example implementations, andin other example implementations, selecting the traffic distributionmode with a combined reliability probability above the targetreliability probability comprises causing a bicasted transmission to bedirected to the cyberphysical system via one subflow associated with thefirst access network and via one subflow associated with the secondaccess network. In some such example implementations, and in otherexample implementations, selecting the traffic distribution mode with acombined reliability probability above the target reliabilityprobability comprises causing a transmission to be directed to thecyberphysical system via one subflow associated with the first accessnetwork and causing a bicasted version of the transmission to bedirected to the cyberphysical system via the second access network. Insome such example implementations, and in other example implementations,selecting a traffic distribution mode with a combined reliabilityprobability above the target reliability probability comprises selectingthe traffic distribution mode with the highest ranking.

BRIEF DESCRIPTION OF THE DRAWINGS

Having thus described certain example embodiments of the presentdisclosure in general terms, reference will hereinafter be made to theaccompanying drawings, which are not necessarily drawn to scale, andwherein:

FIG. 1 depicts an example system environment in which implementations ofexample embodiments of the present invention may be performed;

FIG. 2 is a block diagram of an apparatus that may be specificallyconfigured in accordance with an example embodiment of the presentinvention;

FIG. 3 depicts an example system environment in which aspects ofimplementations of example embodiments of the present invention may beillustrated and/or performed;

FIG. 4 depicts an example system environment in which aspects ofimplementations of example embodiments of the present invention may beperformed;

FIGS. 5A-5F depict traffic distribution modes which may be used inconnection with implementations of example embodiments of the presentinvention;

FIG. 6 is a graphical representation of a performance of a radio nodethat illustrates concepts associated with example embodiments of theinvention;

FIGS. 7A-7F depict annotated versions of FIGS. 5A-5F which provideadditional detail regarding aspects of example embodiments of thepresent invention;

FIG. 8 is a message flow diagram depicting an example implementation ofembodiments of the present invention; and

FIG. 9 is a flowchart illustrating a set of operations performed, suchas by the apparatus of FIG. 2, in accordance with an example embodimentof the present invention.

DETAILED DESCRIPTION

Some embodiments will now be described more fully hereinafter withreference to the accompanying drawings, in which some, but not all,embodiments of the invention are shown. Indeed, various embodiments ofthe invention may be embodied in many different forms and should not beconstrued as limited to the embodiments set forth herein; rather, theseembodiments are provided so that this disclosure will satisfy applicablelegal requirements. Like reference numerals refer to like elementsthroughout. As used herein, the terms “data,” “content,” “information,”and similar terms may be used interchangeably to refer to data capableof being transmitted, received and/or stored in accordance withembodiments of the present invention. Thus, use of any such terms shouldnot be taken to limit the spirit and scope of embodiments of the presentinvention.

Additionally, as used herein, the term ‘circuitry’ refers to (a)hardware-only circuit implementations (e.g., implementations in analogcircuitry and/or digital circuitry); (b) combinations of circuits andcomputer program product(s) comprising software and/or firmwareinstructions stored on one or more computer readable memories that worktogether to cause an apparatus to perform one or more functionsdescribed herein; and (c) circuits, such as, for example, amicroprocessor(s) or a portion of a microprocessor(s), that requiresoftware or firmware for operation even if the software or firmware isnot physically present. This definition of ‘circuitry’ applies to alluses of this term herein, including in any claims. As a further example,as used herein, the term ‘circuitry’ also includes an implementationcomprising one or more processors and/or portion(s) thereof andaccompanying software and/or firmware. As another example, the term‘circuitry’ as used herein also includes, for example, a basebandintegrated circuit or applications processor integrated circuit for amobile phone or a similar integrated circuit in a server, a cellularnetwork device, other network device, and/or other computing device.

As defined herein, a “computer-readable storage medium,” which refers toa non-transitory physical storage medium (e.g., volatile or non-volatilememory device), can be differentiated from a “computer-readabletransmission medium,” which refers to an electromagnetic signal.

A method, apparatus and computer program product are provided inaccordance with example embodiments in order to provide for the dynamicdistribution of network traffic based on reliability probabilities ofindividual subflows within a hybrid network environment. Manyimplementations of example embodiments of the invention described andotherwise disclosed herein are directed to solving the technicalchallenges associated with providing a combined, reliable, and real-timeconnectivity solution for a given combined target over a hybrid networkwhich may consist of several network subflows and/or sub-networks.

Example embodiments of the invention described and/or otherwisedisclosed herein generally relate to the use of hybrid networks that maycombine loosely coupled technologies for the support of mission criticaluse cases. It will be appreciated that some example implementations ofembodiments of the invention are directed to providing networkconnectivity that meets mission-critical reliability and timingrequirements at a lower cost and faster time-to-market than conventionalapproaches. For example, some example implementations are directed tomission-critical applications in scope that demand high networkreliability (for example, up to 99.999%) for a specified real-timerequirement (for example, at a 10-100 ms level). Some such exampleimplementations involve and/or may be referred to as reliable real-timecyber-physical systems (CPS). It will be appreciated that, in manycontexts, CPS systems should be distinguished from 3GPPultra-reliability low-latency communications (URLLC) that target what isoften referred to as “5-nines” reliability with latency down to a 1 mslevel.

As noted herein, many example implementations are particularly directedto addressing technical challenges that arise in contexts involvinghybrid networks. For example, some example implementations ofembodiments of the invention described and/or otherwise disclosed hereininvolve and/or contemplate the combining of legacy 3GPP technologies,such as networks and/or portions of networks that incorporate 2G and/or3G protocols and/or other aspects, with implementations of more recentLTE/NR (5G) radios. Some such example implementations and other exampleimplementations contemplate and/or involve combining the networks of twoor more operators for optimized and/or otherwise improved site andequipment diversity (which may involve multi-homing, for example). Somesuch example implementations and other example implementations involvefurther combining 3GPP access with non-3GPP and non-integrated solutionssuch as Wi-Fi, Multefire, Lora, and possibly even satellite networks,which can be envisioned to realize such a wireless hybrid accessnetwork. In some situations, example embodiments of the inventiondescribed and/or otherwise disclosed herein address technical issuesassociated with extensions of state-of-the-art hybrid accesstechnologies to become capable of delivering wireless, reliable,real-time and/or near real-time connectivity.

FIG. 3 presents an example network environment 300 which may be usefulfor presenting a simplified explanation of some of the technicalchallenges addressed and overcome by example embodiments of theinvention described and/or otherwise disclosed herein. It will beappreciated that example network environment is merely provided toprovide context to and assist in explanations of some aspects of some ofthe technical challenges that arise in at least some hybrid networks.Nothing in FIG. 3 or the related discussion of such technical challengesshould be construed as limiting the technical challenges addressed byembodiments of the invention (or the approaches taken by exampleimplementations of embodiments of the invention) to the particularenvironment depicted in FIG. 3.

As shown in FIG. 3, example environment 300 involves a hybrid networkthat involves at least two underlying access networks: a first accessnetwork 304 and a second access network 306. In some exampleimplementations, the first access network 304 incorporates and/orinvolves a first interface which may be assigned an identifier such asPLMN1 and/or SSID1, for example. Likewise, in some exampleimplementations, the second access network 304 incorporates and/orinvolves a second interface which may be assigned an identifier such asPLMN2 and/or SSID2.

Depending on the situation and context in which an exampleimplementation of environment 300 arises, access networks 304 and 306could potentially belong to different operators (which may be reflectedthrough the use of identifiers such as PLMN1 and/or PLMN2) as well asuse the same or different radio access technologies (for example, LTEPLMN1, WLAN SSID2). It will be appreciated that in many situations, eachaccess network in a hybrid network environment such as environment 300should be considered to be a black box, at least in the sense that auser of a network device that interacts with access networks 304 and 306and/or other access networks within a hybrid environment will typicallyhave little or no control over the underlying structure, function, andoperation of the hybrid network as a whole and/or the access networks304 and 306 within such a network environment.

As shown in FIG. 3, environment 300 involves, at one end, a multihomingor Multi-Path TCP capable device 302. In some example implementations,device 302 is connected simultaneously to both access networks 304 and306. In FIG. 3, device 302 may send and receive information and/orotherwise interact with access network 304 via subflow 308, which, inthe example implementation shown in FIG. 3, may be considered to be aprimary subflow. As also shown in FIG. 3, device 302 may send andreceive information and/or otherwise interact with access network 306via subflow 310, which, in the example implementation shown in FIG. 3,may be considered to be a secondary subflow. In some exampleimplementations, the device 302 is configured to incorporate a trafficrecombination and distribution module 302 a which may interact withmodules 302 b and 302 c, which are configured to transmit and/or receivethe subflows 308 and 310 (as shown in FIG. 3, with module 302 binteracting with subflow 308 and module 302 c interacting with subflow310). Regardless of its exact configuration, device 302 can combine thetraffic of a given application received by the two networks 304 and 306in the downlink direction. Likewise, in the uplink direction, the device302 can split the traffic on a given application across the two networks304 and 306 which is then recombined at the network side at the MPTCPproxy/server or multihoming anchor point 312. In some exampleimplementations, the MPTCP proxy/server 312 is configured to incorporatea traffic recombination and distribution module 312 a which may interactwith modules 312 b and 312 c, which are configured to transmit and/orreceive the subflows 308 and 310, as relayed by their respective accessnetworks 304 and 306 (as shown in FIG. 3, with module 312 b interactingwith subflow 308 via access network 304 and module 312 c interactingwith subflow 310 via access network 306). Regardless of its exactconfiguration, MPTCP proxy/server 312 can combine the traffic of a givenapplication received by the two networks 304 and 306 in the uplinkdirection. Likewise, in the downlink direction, the MPTCP proxy/server312 can split the traffic across the two networks 304 and 306 which isthen recombined at the client side at mobile device 302. As shown inFIG. 3, the MPTCP proxy/sever 312 is configured to also be able tointeract with the Internet and/or other networks 314.

One of the significant technical challenges addressed by exampleembodiments of the invention described and/or otherwise disclosed hereinis the design of an efficient traffic distribution control at thebifurcation point (such as the MPTCP proxy/server 312 shown in FIG. 3,for example). Addressing such a technical challenge involves aspectsrelated to network resource allocation and usage and aspects involvinguser equipment battery, which meeting the wireless reliable real-timeconnectivity targets that may be imposed in a given situation and/orcontext. In particular, example embodiments of the invention describedand/or otherwise disclosed herein address challenged involved with theefficient control of packet distribution over two or more accessnetworks in a manner that guarantees network resource availability andreliability at a specified level (such as up to 99.999%, for example)and for specified real-time requirement (such as at a 10-100 ms level,for example).

It will be appreciated that some approaches attempt to address some ofthe technical challenges that arise in hybrid network environments. Forexample, some multi-connectivity (MC) solutions involve transmission andreception to an end-user device across multiple communication subflowssimultaneously. In such approaches, MC is used primarily to boost datarates, improve mobility performance, and, as a secondary function, toprovide connectivity backup during primary network downtimes. However,none of the existing approaches address specifically reliable real-timerequirements. For example some MP approaches use TCP properties to inferpath properties, namely round-trip time (RTT) estimation, and to makeforwarding decisions with the aim of reducing reordering delays that areknown to impair TCP performance. In some other approaches, every packetof a short flow is replicated and transmitted concurrently acrossmultiple subflows. While such approaches may reduce the retransmissiondelays associated with lost packets, they necessarily create significantoverhead in the network.

It will be appreciated that the term “reliability”, as used herein,conforms to the definition contained in 3GPP (TR 38.913) as a compositemetric of latency and packet loss ratio, namely, the probability totransfer successfully X bytes within a certain delay budget.

Example embodiments of the invention described and/or otherwisedisclosed here address these and other technical challenges by providingan approach for dynamic traffic distribution based at least in part onreliability probabilities of the single subflows. As such, exampleembodiments of the invention also provide a combined reliable realtimeconnectivity solution operating to a given combined target over a hybridnetwork consisting of several network subflows (or sub networks). Whilesome of the example implementations discussed herein involve orcontemplate availability and/or reliability up to 99.999% at a specifiedlatency of less than 100 ms, it will be appreciated that exact and/orother targets are configurable. Some example implementations ofembodiments of the invention are particularly advantageous when used inconnection with wide area cyber physical systems (CPS), including butnot limited to systems associated with autonomous vehicles such asdrones, trains, cars as well as industrial applications, for example.

In some example implementations, embodiments of the invention providefor and use the prediction capability of inferring the likelihood that arelevant reliability target can be met for both the individual subflowswithin a hybrid environment and for the overall hybrid environment. Insome example implementations, the approach taken to provide and use suchcapabilities involves the following:

(1) The computation of reliability probabilities on a per subflow (whichalso referred to as link) basis. In some situations, the reliabilityprobabilities indicate the likelihood of correct data reception at L3/L4(such as in situations involving TCP/UDP/IP, for example) within a givendelay for the given subflow;

(2) The signaling of such reliability probabilities and/or relatedmetrics to the control system for the hybrid network, such as viapredefined interface and signaling protocol, for example. In some of theexamples discussed herein, the control system may be referred to as atraffic distribution entity;

(3) At the control system or traffic distribution entity, thecomputation of a combined reliability probability for differentconfigurations of traffic distribution modes (for example,single-network-only, single-network-only plus selected packetduplication over the other network, bi/multi-casting serially,bi/multi-casting in parallel); and

(4) The dynamic selection of the preferred traffic distribution modefrom among the modes which meet a pre-defined reliability probabilitytarget. In some situations, this may involve a sequence of L3/L4packets, with the objective of minimizing a defined cost function (suchas the cost of network/bandwidth use, energy, network resource, and/orpenalties from not meeting the reliability target, or the like forexample).

Some example implementations of embodiments of the invention take aniterative approach to dynamically distributing traffic within a hybridnetwork environment. In such implementations, a traffic distributionentity queries and/or receives one or more reliability probabilities pereach relevant link in the hybrid network. In some situations, thesereliability probabilities may be marked as a ReliabilityProbability orRP. Upon receipt of the reliability probabilities, the trafficdistribution entity computes a combined reliability probability forvarious different configurations of traffic distribution modes. In somesituations, these combined reliability probabilities may be marked as acombined ReliabilityProbability or combinedRP. After the combinedreliability probabilities are calculated, the traffic distributionentity ranks each traffic distribution mode for which the relevantcombined reliability probability is higher than a target reliabilityprobability. In some situations, a pre-defined rule set, such as a costfunction, for example, may be used in connection with ranking thecombined reliability probabilities. The traffic distribution entity maythen select the best traffic distribution mode (based at least in parton the ranking). These steps may be iteratively performed by a trafficdistribution entity at predetermined time intervals, in response to thedetection of one or more events and/or occurrences, and/or inconformance with predetermined rules established in connection with thetraffic distribution entity.

While the method, apparatus and computer program product of an exampleembodiment may be deployed in a variety of different systems, oneexample of a system that may benefit from the procedures discussed andcontemplated herein in accordance with an example embodiment of thepresent invention is depicted in FIG. 1. The depiction of systemenvironment 100 in FIG. 1 is not intended to limit or otherwise confinethe embodiments described and contemplated herein to any particularconfiguration of elements or systems, nor is it intended to exclude anyalternative configurations or systems for the set of configurations andsystems that can be used in connection with embodiments of the presentinvention. Rather, FIG. 1, and the system environment 100 disclosedtherein is merely presented to provide an example basis and context forthe facilitation of some of the features, aspects, and uses of themethods, apparatuses, and computer program products disclosed andcontemplated herein. It will be understood that while many of theaspects and components presented in FIG. 1 are shown as discrete,separate elements, other configurations may be used in connection withthe methods, apparatuses, and computer programs described herein,including configurations that combine, omit, and/or add aspects and/orcomponents.

As shown in FIG. 1, the system environment includes one or more userequipment devices 102 configured to communicate wirelessly, such as viaone or more access networks, with a network 106. Although the userequipment 102 may be configured in a variety of different manners, theuser equipment may be embodied as a mobile terminal, such as a portabledigital assistant (PDA), mobile phone, smartphone, pager, mobiletelevision, gaming device, laptop computer, camera, tablet computer,communicator, pad, headset, touch surface, video recorder, audio/videoplayer, radio, electronic book, positioning device (e.g., globalpositioning system (GPS) device), or any combination of theaforementioned, and other types of voice and text and multi-modalcommunications systems.

In addition to the more traditional types of user equipment 102 whichmay be present within a given system environment, system environment 100also includes one or more devices 110 that are capable of interactingwith a system environment, such as through the use of wirelesscommunication. In some example implementations, one or more of thedevices 110 may be incorporated into and/or otherwise associated with anInternet-of-Things (IoT) user equipment device, which may be referred toas an IoT device. In some example implementations, one or more devices110 may be incorporated into and/or otherwise associated with acyber-physical system, which may include but are not limited toautonomous vehicles (such as drones, trains, cars, and the like, forexample) and/or industrial equipment, for example. Although the userequipment device 102 and the device 110 may be configured in a varietyof different manners, it will be appreciated that in many of the exampleimplementations discussed and otherwise contemplated herein the relevantdevice 102 and/or relevant device 110 will be configured to operate in ahybrid network, including but not limited to interacting with multipleaccess networks which may be available to a given device 102 and/ordevice 110.

System environment 100, as depicted in FIG. 1, also includes one or moreaccess points 104 a and 104 b, such as base stations (such as node Bs,evolved Node Bs (eNB), or the like, for example). A cellular accesspoint, such as a base station, may define and service one or more cells.The access points may, in turn, be in communication with a network 106,such as a core network via a gateway, such that the access pointsestablish cellular radio access networks by which the user equipment 102and/or sensors 110 may communicate with the network. The systemenvironment 100 of FIG. 1 may include a plurality of different cellularradio access networks including, for example, a 5G radio access network,an LTE radio access network, a UMTS (universal mobile telecommunicationssystem) radio access network, etc. In some example implementations,equipment and other infrastructure associated with multiple differentcellular radio access networks may be located at or near structuresand/or other equipment associated with a particular access point, suchas access point 104 a and 104 b.

In some implementations of system environment 100, the cellular radioaccess networks serviced by access points 104 a, 104 b, and any otheraccess points in a given area are identical, in the sense that as userequipment 102 and/or sensor 110 moves from an area serviced by accesspoint 104 a to an area serviced by access point 104 b, the userequipment 102 and/or sensor device 110 is able to access the network 106via a radio access network provided by the same vendor across accesspoints. In some implementations of system environment 100, the cellularradio access networks serviced by access points 104 a and 104 b may bedifferent and/or part of a hybrid network that incorporates networkelements conforming to different standards or protocols, incorporatesnetwork elements belonging to different vendors and/or operators, and/orincorporates network elements that are otherwise loosely coupled and/orrelatively independent of each other. Although not shown, the system mayalso include one or more controllers associated with one or more of thecellular access points, (such as base stations for example), so as tofacilitate operation of the access points and management of the userequipment 102 and/or sensor 110 in communication therewith. As shown inFIG. 1, a system may also include one or more wireless local areanetworks (WLANs), each of which may be serviced by a WLAN access point108 configured to establish wireless communications with the userequipment 102 and/or the sensor 110. As such, the user equipment 102and/or the sensor 110 may communicate with the network via a WLAN accesspoint as shown in solid lines in FIG. 1, or, alternatively, via acellular access point as shown in dashed lines. The radio accessnetworks as well as the core networks may consist of additional networkelements as routers, switches, servers, gateways, and/or controllers.

In connection with the approaches to provide dynamic trafficdistribution based on reliability probabilities of the individualsubflows within a hybrid network environment, some exampleimplementations of embodiments of the invention disclosed and/orotherwise described herein contemplate the use of network entities suchas servers (including but not limited to cloud servers, for example),gateways, and/or other network elements that operate as a controlsystem, which may be referred to herein as a traffic distributionentity. In this regard, the calculation, processing, and use of networksubflow reliability profiles to dynamically direct and redirect networktraffic can be accomplished by an apparatus 200 as depicted in FIG. 2.The apparatus may be embodied by and/or incorporated into one or moreservers, such as a server associated with network 106, or any of theother devices discussed with respect to FIG. 1, such as access points104 a and/or 104 b, one or more of WLAN access points 108, and/ordevices that may be incorporated or otherwise associated with systemenvironment 100. The apparatus may be embodied by and/or incorporatedinto any of the devices discussed in connection with FIG. 3 or 4,including but not limited to the multihoming anchor/MPTCP proxy/server312 and/or MPTCP proxy 412. Alternatively, the apparatus 200 may beembodied by another device, external to such devices. For example, theapparatus may be embodied by a computing device, such as a personalcomputer, a computer workstation, a server or the like, or by any ofvarious mobile computing devices, such as a mobile terminal, (such as asmartphone, a tablet computer, or the like, for example). In someexample implementations, it may be particularly advantageous toimplement the apparatus 200 in connection with one or more networkelements and/or functions.

Regardless of the manner in which the apparatus 200 is embodied, theapparatus of an example embodiment is configured to include or otherwisebe in communication with a processor 202 and a memory device 204 andoptionally the user interface 206 and/or a communication interface 208.In some embodiments, the processor (and/or co-processors or any otherprocessing circuitry assisting or otherwise associated with theprocessor) may be in communication with the memory device via a bus forpassing information among components of the apparatus. The memory devicemay be non-transitory and may include, for example, one or more volatileand/or non-volatile memories. In other words, for example, the memorydevice may be an electronic storage device (such as a computer readablestorage medium, for example) comprising gates configured to store data(such as bits, for example) that may be retrievable by a machine (suchas a computing device like the processor, for example). The memorydevice may be configured to store information, data, content,applications, instructions, or the like for enabling the apparatus tocarry out various functions in accordance with an example embodiment ofthe present invention. For example, the memory device could beconfigured to buffer input data for processing by the processor.Additionally or alternatively, the memory device could be configured tostore instructions for execution by the processor.

As described above, the apparatus 200 may be embodied by a computingdevice. However, in some embodiments, the apparatus may be embodied as achip or chip set. In other words, the apparatus may comprise one or morephysical packages (such as chips, for example) including materials,components and/or wires on a structural assembly (such as a baseboard,for example). The structural assembly may provide physical strength,conservation of size, and/or limitation of electrical interaction forcomponent circuitry included thereon. The apparatus may therefore, insome cases, be configured to implement an embodiment of the presentinvention on a single chip or as a single “system on a chip.” As such,in some cases, a chip or chipset may constitute means for performing oneor more operations for providing the functionalities described herein.

The processor 202 may be embodied in a number of different ways. Forexample, the processor may be embodied as one or more of varioushardware processing means such as a coprocessor, a microprocessor, acontroller, a digital signal processor (DSP), a processing element withor without an accompanying DSP, or various other processing circuitryincluding integrated circuits such as, for example, an ASIC (applicationspecific integrated circuit), an FPGA (field programmable gate array), amicrocontroller unit (MCU), a hardware accelerator, a special-purposecomputer chip, or the like. As such, in some embodiments, the processormay include one or more processing cores configured to performindependently. A multi-core processor may enable multiprocessing withina single physical package. Additionally or alternatively, the processormay include one or more processors configured in tandem via the bus toenable independent execution of instructions, pipelining and/ormultithreading.

In an example embodiment, the processor 202 may be configured to executeinstructions stored in the memory device 204 or otherwise accessible tothe processor. Alternatively or additionally, the processor may beconfigured to execute hard coded functionality. As such, whetherconfigured by hardware or software methods, or by a combination thereof,the processor may represent an entity (such as by being physicallyembodied in circuitry, for example) capable of performing operationsaccording to an embodiment of the present invention while configuredaccordingly. Thus, for example, when the processor is embodied as anASIC, FPGA or the like, the processor may be specifically configuredhardware for conducting the operations described herein. Alternatively,as another example, when the processor is embodied as an executor ofsoftware instructions, the instructions may specifically configure theprocessor to perform the algorithms and/or operations described hereinwhen the instructions are executed. However, in some cases, theprocessor may be a processor of a specific device (such as apass-through display or a mobile terminal, for example) configured toemploy an embodiment of the present invention by further configurationof the processor by instructions for performing the algorithms and/oroperations described herein. The processor may include, among otherthings, a clock, an arithmetic logic unit (ALU) and logic gatesconfigured to support operation of the processor.

In some embodiments, the apparatus 200 may optionally include a userinterface 206 that may, in turn, be in communication with the processor202 to provide output to the user and, in some embodiments, to receivean indication of a user input. As such, the user interface may include adisplay and, in some embodiments, may also include a keyboard, a mouse,a joystick, a touch screen, touch areas, soft keys, a microphone, aspeaker, or other input/output mechanisms. Alternatively oradditionally, the processor may comprise user interface circuitryconfigured to control at least some functions of one or more userinterface elements such as a display and, in some embodiments, aspeaker, ringer, microphone and/or the like. The processor and/or userinterface circuitry comprising the processor may be configured tocontrol one or more functions of one or more user interface elementsthrough computer program instructions (such as software and/or firmware,for example) stored on a memory accessible to the processor (such asmemory device 204, and/or the like, for example).

The apparatus 200 may optionally also include the communicationinterface 208. The communication interface may be any means such as adevice or circuitry embodied in either hardware or a combination ofhardware and software that is configured to receive and/or transmit datafrom/to a network and/or any other device or module in communicationwith the apparatus. In this regard, the communication interface mayinclude, for example, an antenna (or multiple antennas) and supportinghardware and/or software for enabling communications with a wirelesscommunication network. Additionally or alternatively, the communicationinterface may include the circuitry for interacting with the antenna(s)to cause transmission of signals via the antenna(s) or to handle receiptof signals received via the antenna(s). In some environments, thecommunication interface may alternatively or also support wiredcommunication. As such, for example, the communication interface mayinclude a communication modem and/or other hardware/software forsupporting communication via cable, digital subscriber line (DSL),universal serial bus (USB) or other mechanisms.

As noted herein, many implementations of example embodiments of theinvention described, contemplated, and/or otherwise disclosed herein aredirected to solutions that allow for the dynamic distribution of trafficin a hybrid network environment based at least in part on thecalculation, analysis, and use of reliability probabilities of theindividual subflows available within the hybrid network. Some suchexample implementations allow for the provision of a combined, reliable,real time or near-real time connectivity solution that may be used toensure that connectivity between a cyberphysical system and a hybridnetwork meet relevant availability and/or reliability thresholds atcertain latency requirements.

FIG. 4 provides an example network environment 400 in which aspects ofembodiments of the invention may be illustrated. For purposes of clarityand simplicity, the examples discussed in connection with FIG. 4 assumethe use of MPTCP aspects and involve two underlying networks within thehybrid network environment 400. However, it will be appreciated that theconcepts discussed herein, and the invention described and/or otherwisedisclosed herein, are applicable also when assuming any other transportmodel mechanisms to transport traffic between the traffic distributionentity and the client (including but not limited to tunneling, forexample). It will also be appreciated that the invention describedand/or otherwise disclosed herein may be implemented in situationsinvolving more than two subnetworks (or sub-flows or links), dependingon their availability within a given hybrid network environment.Moreover, it will be appreciated that example implementations can alsobe applicable for single link systems as well as when more than twonetworks are present.

As shown in FIG. 4, example environment 400 involves a hybrid networkthat involves at least two underlying access networks: a first accessnetwork 404 and a second access network 406. In some exampleimplementations, the first access network 404 incorporates and/orinvolves a first interface which may be assigned an identifier such asPLMN1 and/or SSID1, for example. Likewise, in some exampleimplementations, the second access network 304 incorporates and/orinvolves a second interface which may be assigned an identifier such asPLMN2 and/or SSID2, for example.

It will be appreciated that first and second access networks 404 and 406are similar to access networks 304 and 306 discussed herein with respectto FIG. 3, at least in the sense that, depending on the situation andcontext in which an example implementation of environment 400 arises,access networks 404 and 406 may belong to different operators as well asuse the same or different radio access technologies. Regardless of theprecise configurations and relationships between access networks 404 and406, both access networks 404 and 406 are configured to be capable ofcommunicating with a user device 402.

Example hybrid environment 400 involves, at one end, a multihoming orMulti-Path TCP (MPTCP) capable user device 402. In the example depictedin FIG. 4, device 402 is connected simultaneously to both accessnetworks 404 and 406. As such, device 402 may send and receiveinformation and/or otherwise interact with access network 404 viasubflow 408, which, in the example implementation shown in FIG. 4, maybe considered to be a primary TCP subflow. As also shown in FIG. 4,device 402 may send and receive information and/or otherwise interactwith access network 406 via subflow 410, which, in the exampleimplementation shown in FIG. 4, may be considered to be a secondary TCPsubflow.

As depicted in FIG. 4, the device 402 is configured to incorporate atraffic recombination and distribution module 402 a which may interactwith modules 402 b and 402 c, which are configured to transmit and/orreceive the subflows 408 and 410 (as shown in FIG. 4, with module 402 binteracting with subflow 408 and module 402 c interacting with subflow410). Regardless of its exact configuration, device 402 can combine thetraffic of a given application received by the two networks 404 and 406in the downlink direction. Likewise, in the uplink direction, the device402 can split the traffic on a given application across the two networks404 and 406 which is then recombined at the network side at the MPTCPproxy 412. While in the example depicted in FIG. 4 shows user device 402to be an MPTCP device, it will be appreciated that other exampleimplementations of hybrid network 400 and other embodiments of theinvention may involve other user devices that are capable of interactingwith multiple subflows or other communication streams in a hybridnetwork environment.

As shown in FIG. 4, the MPTCP proxy 412 is configured to incorporate atraffic recombination and distribution module 412 a (which may bereferred to as a traffic distribution entity), which may incorporateand/or interact with modules 412 b and 412 c, which are configured totransmit and/or receive the subflows 408 and 410, as relayed by theirrespective access networks 404 and 406. As shown in FIG. 4, module 412 binteracts with subflow 408 via access network 404 and module 412 cinteracts with subflow 410 via access network 406). Regardless of itsexact configuration, MPTCP proxy 412 can combine the traffic of a givenapplication received by the two networks 404 and 406 in the uplinkdirection. Likewise, in the downlink direction, the MPTCP proxy 412 candetermine how to split the traffic across the two networks 404 and 406which is then recombined at the client side at mobile device 402. Asshown in FIG. 4, the MPTCP proxy 412 is configured to also be able tointeract with the Internet and/or other networks 414.

As shown in FIG. 4, in addition to receiving application-relatedinformation, user generated information, and/or other communicationspayload information via access networks 404 and 406, the MPTCP proxy 412is also configured to receive reliability probability information 416from access network 404 and reliability probability information 418 fromaccess network 406. In some example implementations the reliabilityprobabilities 416 and 418 are a composite metric of a relevant latencyand a relevant packet loss ratio, such that it represents thatprobability to successfully transfer a certain number of bytes and/orother quantity of data within a certain delay budget or other timingparameter.

As noted herein, MPTCP proxy 412, through the operation of the trafficrecombination and distribution module 412 a and modules 412 b and 412 c,functions as a traffic distribution entity. In some exampleimplementations, traffic recombination and distribution module 412 aand/or other components of MPTCP proxy 412 provide for predictivetraffic distribution by: (1) monitoring the relevant subflows, which mayinclude monitoring their respective reliability probability metricsand/or other indications of reliability, such as TCP acknowledgement(ACK) or non-acknowledgment (NACK) messages; (2) dynamic evaluation ofthe available traffic distribution modes based at least in part on acomparison of a combination of the relevant reliability probabilitymetrics with a target reliability probability metric; and (3)potentially adjusting the traffic distribution mode. In some exampleimplementations, an adjustment to the traffic distribution mode may beassociated with a reliability-based trigger, such as an indication thatthe communication reliability is likely to fall short of a targetreliability threshold.

FIGS. 5A-5F depict a series of multiple different transmission modesthat may be used in example environments (such as example hybridenvironment 400, for example) that involve two access networks,subflows, and/or links, which are marked in FIG. 5 as access network 502and access network 504, respectively. For the purposes of clarity, asolid line represents a single transmission within a given network, adouble line represents bicasting of the same packet or sequence ofpatents, and a dotted line represents no transmission. It will beappreciated that the example modes presented in FIGS. 5A-5F presentnon-limiting examples of the transmission modes that may be used in agiven network environment, and that other transmission modes may be useddepending on the precise implementation of the hybrid network and thecapabilities of the underlying component networks therein. Moreover, itwill be appreciated that the distribution modes presented in FIG. 5A-5Fcan be expanded to a larger number of combinations assuming theavailability of more than two access networks, and may be generalized toany number of links, as well.

FIG. 5A depicts a transmission mode where a single transmission 502 a ismade via access network 502, and no transmission is made via accessnetwork 504.

FIG. 5B depicts a transmission mode where a single transmission 504 a ismade via access network 504, and no transmission is made via accessnetwork 502.

FIG. 5C depicts a transmission mode that involves bicasting in parallelwhere a single transmission 502 a is made via access network 502, and asingle transmission 504 a is made via access network 504.

FIG. 5D depicts a transmission mode that involves serial bicasting,where two transmissions 502 a and 502 b are made via access network 502,and no transmission is made via access network 504.

FIG. 5E depicts a transmission mode that involves serial bicasting,where two transmissions 504 a and 504 b are made via access network 504,and no transmission is made via access network 502.

FIG. 5F depicts a transmission mode that involves serial and parallelbicasting, where one transmission 502 a is made via access network 502,and two transmissions 504 a and 504 b are made via access network 504.

The Subflow Reliability Metric

As noted herein, some example implementations of embodiments of theinvention involve the determination, processing, and use of one or moresubflow reliability metrics. For example, in order for the trafficdistribution function to determine the optimal number of subflows to useand their configuration (such as to select between the varioustransmission distribution modes depicted in FIGS. 5A-5F, for example),each subflow reports a probability reliability metric. In some exampleimplementations, the reliability metric indicates the likelihood of apacket (at an L3/L4 level, for example) to be correctly received withina given delay budget. In some instances, this may be expressed as aprobability indicated by a percentage value, for example. The value mayalso have a time-domain component. For example the value may be of arelatively semi-static nature (for example, where it is averaged over asignificant time window and thus apply for a longer duration of time) orcan be of a more instantaneous and dynamic nature (for example where itrelates to a short time-frame and may only be valid for a shorttime-frame in the near future).

In an example implementation, the delay budget for which the metric isobtained can be configured by the traffic distribution unit so that thesubflow provides one or more optimal metric estimates tailored for thespecific application or use-case. In other example implementations, thedelay metric may be a more generic value (and/or always reported formultiple values) such as where the application needs to extrapolateand/or otherwise estimate the total relevant application latencyrequirement.

Depending on the situation in which an example implementation arises,particularly with respect to the nature of the transmissions and thecontexts in which the various network elements operate, an example valueof the reliability probability metric may be, for example, 90%, whichwould indicate that there is a 90% probability that a packet or acertain sequence of packets is correctly received by the end pointwithin the delay budget (which could be configured to 50 ms, forexample).

In some example implementations, each subflow (which could involve, forexample, a full access network) may apply information and/or otherintelligence from all of the relevant network components to achieve atotal reliability estimate for the whole subflow. For example, such atotal reliability estimate may take into account core routing delays,load and congestion situations in the transmission network, and/or loadand signal quality in the radio network. Regardless of the informationused to develop the reliability network, the metric relates to thesubflow towards a specific end-user. As such, the metric is reported forand should be valid for the end-user device or IoT device for whichreliable realtime communication is being requested. In some exampleimplementations and situations, the network may simplify its proceduresto generate reliability metrics that are applicable for a certaingeographical zone or a certain set of end-users or IoT devices.

One particular technical challenge associated with establishing reliablerealtime communications over cellular subflows is the dynamics of theradio channel. For example, the reliability of the radio channel can beimpacted due to handover events (and resulting gaps), placement andcapabilities of the device at the receiving end, as well as the load andinterference caused by multiple users accessing the same system, forexample. As such, depending on the situation in which a particularembodiment is implemented or deployed, radio network elements will oftenplay a significant role in contributing to the reliability metric. Forexample, a base station (or the modem component on the device end) maytake into account several radio factors in developing a reliabilitymetric: (1) radio link quality indicators (such as UE'sSINR/RSRP/RSRQ/RSSI, or the like, for example); (2) retransmissions(such as L1/L2 ACK/NACK of HARQ process and MAC/RLC ACK/NACK, forexample); (3) the relevant cell load, for example, and (4) other radioreliability factors.

In some cases, the reliabilities of other components compared against aspecific realtime reliability requirement (which may involvetransmission and routing delays, core and gateway processing delays, forexample) may be negligible, such that the reliability metric can beestimated based primarily (or, in some instances, entirely) onreliability estimates provided within and/or associated with the radionetwork layer.

FIG. 6 depicts a graph 600 plotting the cumulative distribution function(CDF) 602 of a radio link. As shown in FIG. 6, the x-axis 604 representstime, with an example indication of 20 ms marked at position 604 a. They-axis 606 in FIG. 6 represents the reliability probability of the radiolink, with an example indication of an 85% reliability probabilitymarked at position 606 a. In the example shown in FIG. 6, an estimatingnode has been collecting statistics of the correctly received packetsduring a preceding time interval (such as during the last second, forexample). Based on this information it can, for example, be determinedthat there an 85% reliability probability of correct packet receptionwithin 20 ms (which may be a configured target for a reliability metric,for example). In some implementations, the metric associated with theradio link may be updated as additional time passes and/or additionalinformation becomes available.

Reporting, Exchange, and Signaling of the Reliability Metric

Example embodiments of the invention described and/or otherwisedisclosed herein involve the use of a reliability metric to provide atleast a partial basis for a determination of a transmission mode to beused in a hybrid network environment. For the traffic distributionfunctionality to leverage the reliability metric, the reliability metricmust be communicated from the relevant subflow layer to the trafficcontrol entity and/or another network element that is able to controland/or direct the transmission mode used within the hybrid networkenvironment. It will be appreciated that many approaches may be taken tocommunicating the relevant reliability metrics to the traffic controlentity and/or other relevant network elements, and the optimal approachwill vary and/or depend on the particulars of the situation and/ornetwork architecture implicated in a given implementation. Oneconsideration that impact the approach taken to communicating thereliability probability metric involves whether the traffic distributioncontrol is entirely centralized (for example, configuring both uplink ordownlink transmission) or if part of the traffic distribution control isdistributed to the end-user device or IoT device (for example, forautonomously selecting the optimal approach for uplink distribution).For the purposes of simplicity and clarity, some of the examplesdescribed herein assume that the traffic distribution system may beviewed as a single system (from the sub-flow perspective) and that thehybrid network environment permits the sub-flow(s) to exchange thereliability metric to the traffic distribution unit (which may thendistribute the metric value to its sub-units).

In some example implementations, the reliability metric can be reportedon request, periodically, or based on the occurrence of an event. Forexample, an event can, for instance, be the exceeding of a certain valuein the reliability metric or a significant change to last reportedvalue. In some such example implementations and in other exampleimplementations, the traffic distribution unit may incorporate aninterface to configure how the reliability metric is defined within thesubflow. When setting up the connection, part of the setup process mayinvolve defining which end device the reliability metric is intended toapply to.

Leveraging the Reliability Metric

After receiving the relevant reliability probability metrics associatedwith the available individual (or single) subflows, the trafficdistribution entity will calculate the combined reliability probabilitymetric for the different traffic distribution options. In the exampleprovided below, the combined reliability probability metric is denotedas RPconfn, and can be calculated for the entire set of individualreliability metrics or for a subset of possible traffic distributionconfigurations to predict reliability:

RPconfn (u, t)=f(RP1 (u,t), RP2(u,t)) (for a non-limiting example of twosubflows)

It will be appreciated that “f” is a function of the differentreliability probability metrics for a given user, bearer, and/or packet“u” at the time “t”. The term “RPx” is used to indicate the reliabilitymetric provided by subflow “x” that relates to the relevant u and tvalues.

FIGS. 7A-7F present annotated versions of FIGS. 5A-5F wherein thesubflows associated with access network 502 have been determined to havea reliability probability RP1=90% and the subflows associated withaccess network 504 have been determined to have a reliabilityprobability RP2=80%. In the examples presented in FIGS. 7A-7F, it isassumed that the subflows are uncorrelated and (for the purposes ofsimplicity and clarity) that serially bicasted packets are alsouncorrelated. It will be appreciated that in some other exampleimplementations, however, serially bicasted packets will be likelycorrelated and thus a correlation factor may need to be included in thecalculation of a combined reliability probability RPconfn.

As shown in FIG. 7A, the transmission mode involves a singletransmission 502 a made via access network 502, and no transmission ismade via access network 504. Consequently, RPconfn is the reliabilityprobability of the only subflow used, which in this case is 90%.

As shown in FIG. 7B, the transmission mode involves a singletransmission 504 a made via access network 504, and no transmission ismade via access network 502. Consequently, RPconfn is the reliabilityprobability of the only subflow used, which in this case is 80%.

As shown in FIG. 7C, the transmission mode involves bicasting inparallel where a single transmission 502 a is made via access network502, and a single transmission 504 a is made via access network 504.RPconfn may be calculated as:

RPconfn=1−(1−0.8)*(1−0.9)=98%

As shown in FIG. 7D, the transmission mode involves serial bicasting,where two transmissions 502 a and 502 b are made via access network 502,and no transmission is made via access network 504. RPconfn may becalculated as:

RPconfn=1−(1−0.9)*(1−0.9)=99%

As shown in FIG. 7E, the transmission mode involves serial bicasting,where two transmissions 504 a and 504 b are made via access network 504,and no transmission is made via access network 502. RPconfn may becalculated as:

RPconfn=1−(1−0.8)*(1−0.8)=96%

As shown in FIG. 7F, the transmission mode involves serial and parallelbicasting, where one transmission 502 a is made via access network 502,and two transmissions 504 a and 504 b are made via access network 504.RPconfn may be calculated as:

RPconfn=1−(1−0.9)*(1−0.8)*(1−0.08)=99.6%

In FIG. 7, it can be seen that if the reliability associated with agiven communication pathway to a device must be 99%, for example (suchas if the relevant reliability probability target was set at 99%, forexample) only two of the example configurations shown in FIGS. 7A-7Fmeet the requirement, namely, the serial bicasting on access network 502shown in FIG. 7D and the serial bicasting on access network 504 plusbicasting on access network 502 shown in FIG. 7F. In some exampleimplementations, the configuration shown in FIG. 7D would be selectedbased at least in part on its likely having lower cost (since the numberof packets to be transmitted and the number of involved networks islower than that required in the configuration shown in FIG. 7F.

It should be appreciated that in the calculation of the reliabilitymetric for bicasting options, some assumptions may be need to be maderegarding any correlation between different transmissions of a givenpacket. For example, for a single subflow, the duplication of a packetwithin a very short interval may lead to either loss of both packets orsuccess of both packets in all or nearly all instances. In such anexample of correlated transmission, bicasting may have limited valueoverall. However, as each subflow is generally assumed to be unaware ofhow the traffic distribution unit operates, some of the examplesdescribed herein will assume that such correlation estimates arecalculated and estimated within the traffic distribution unit andupdated over time monitoring bicasting performance.

In general, the selection of the preferred traffic distribution mode,among the modes which meet a pre-defined reliability probability target(such as a reliability target associated with a sequence of L3/L4packets, for example) will be selected with the objective of minimizinga defined cost function (which may include, for example, the costs ofnetwork/bandwidth use, energy, network resources, and/or the like, forexample).

FIG. 8 depicts a message flow 800 that provides an example illustrationof a signaling flow that may occur in accordance with exampleembodiments of the invention. As shown in FIG. 8, message flow 800involves the exchange of messages between a user equipment device 802, afirst access network 804, a second access network 806 and an MPTCPtraffic distribution entity 808. In some example implementations ofmessage flow 800, the network entities 802, 804, 806, and 808 may bearranged in a manner similar to those of the corresponding entities inFIGS. 3 and 4. The example presented in FIG. 8 illustrates a switch froma “single-interface-only” mode to a “duplicate failed packets on theother interface” mode and a “bi-casting in parallel” mode triggered bythe indication that some TCP segments may be failing to meet therelevant latency and/or reliability requirements due to lower layerfailures. In such an example, the packets duplicated on the otherinterface may be sent with a tag “high priority/low latency” and/or asimilar indication.

As shown in FIG. 8, message flow 800 commences with messages 811 and812, which may take the form of queries from the MPTCP trafficdistribution entity 808 to access network 804 and access network 806,respectively. Upon receipt of a query seeking a reliability probability,the access networks 804 and 806 estimate and/or otherwise determinetheir individual reliability probabilities (RP) metrics associated withthe user equipment device 802. In some example implementations, this mayinvolve determining reliability probability metrics based onmeasurements associated with the user equipment device 802 and/or otheractive user equipment devices. In connection with the transmission ofthe queries to access networks 804 and 806, the MPTCP trafficdistribution entity 808 may attempt to assess the initial and/or currenttraffic distribution mode used in connection with the user equipmentdevice 802. This assessment may be based, for example, on historicaldata, a reliability probability metric received in connection with userequipment device 802, and/or a default setting.

As shown in FIG. 8, message flow 800 continues with message 813, wherethe user equipment device 802 establishes a connection with the MCTCPtraffic distribution entity 808. In the example depicted in FIG. 8, theMPTCP traffic distribution entity 808 applies a “single interface only”traffic distribution mode as the initial mode to be used with the userequipment device 802. As shown at message 814, TCP segments 1-N may besent over one subflow to the access network 804. As shown at message815, those same TCP segments 1-N are scheduled and transmitted to theuser equipment device 802. Upon receipt of the TCP segments, the userequipment device 802 may combine the received TCP subflow(s) into asingle TCP connection according to MPTCP mechanisms.

As shown at message 816, the user equipment device 802 may send L1/L2ACK messages back to the access network 804, and, as shown at message817, may send one or more TCP segment(s) ACK indications back to theMPTCP traffic distribution entity.

As time progresses, the first access network 804 may periodicallyprovide an updated reliability probability RP1 to the MPTCP trafficdistribution entity 808, as shown at message 818. Likewise, the secondaccess network 806 may provide an updated reliability probability RP2 tothe MPTCP traffic distribution entity 808, as shown at message 819. Inthe example depicted in FIG. 8, upon receipt of the updated reliabilityprobabilities, the MPTCP traffic distribution entity 808 may assess thereliability provided by the current traffic distribution mode anddetermine that the current mode satisfies the applicable reliabilitymetrics associated with a given situation. Consequently, and as shown atmessage 820, the MPTCP traffic distribution entity 808 may continue touse the single interface only distribution mode to send TCP segmentsN+1−M over the subflow associated with access network 804, which, asshown in message 821, are then scheduled and set by access network 804to the user equipment device 802. As discussed in connection withmessage 815, the user equipment device 802 may combine the received TCPsubflow(s) into a single TCP connection according to MPTCP mechanisms.

In some example implementations, such as the example shown in FIG. 8,the user equipment device 802 may not properly receive all or some ofthe packets associated with a given transmission. As shown in message822, the user equipment device 802 may transmit an L1/L2 NACK message tothe access network 804, which in turn transmits message 823 (which mayinclude an updated reliability probability RP1 that reflect the NACKevent) to the MPTCP traffic distribution entity 808. Upon receipt ofmessage 823, the MPTCP traffic distribution entity 808 assesses thecombined reliability probability of the current traffic distributionmode. Upon determining that the current mode likely does not satisfy therelevant requirements (for example, that the reliability of the currentmode may fall below a target value), the MPTCP traffic distributionentity evaluates alternative modes. In the example depicted in FIG. 8,the MPTCP traffic distribution entity selects and/or assigns a bicast inparallel mode to be used in connection with user equipment device 802,such that a replica of the failing TCP segments are transmitted to theuser device 802 over the second access network 806.

As shown in FIG. 8, message flow 800 continues at message 824, where TCPsegments N+1−M are sent over a subflow associated with access network806. In some implementations of message 824, the TCP segments may beflagged a “high priority/low latency” in order to receive prioritizedtransmission, scheduling, and/or other treatment by the relevant networkelements. Access network 806 then schedules the TCP segments N+1−M fordelivery to the user equipment device 802, as shown in message 825. Insome situations, it is possible that a TCP NACK indication for the TCPsegments failing over access network 804 will not be triggered, as theduplicated segments are received earlier. As shown at message 826, uponreceipt of the TCP segments N+1−M from access network 806, the userequipment device 802 may transmit an L1/L2 ACK indication to accessnetwork 806. Upon receipt of such an indication, the access network 806may map any flagged segments to a highest priority QoS flow and treatsuch flagged segments with priority. As shown at message 827, themessage flow 800 may conclude upon the transmission of a TCP segment(s)ACK indication to the MPTCP traffic distribution entity 808.

In some example implementations of embodiments of the invention that canbe performed in an environment similar to those provided in FIGS. 3, 4,and 8, the traffic distribution entity and relevant access networkmonitors each subflow where data was earlier sent to estimate thereliability metric associated with a given user equipment device. Insome such example implementations, and in other example implementations,the traffic distribution entity and access network probes each subflowwhere data was not sent earlier to estimate the relevant reliabilitymetric. In cases where the reliability metric per subflow is notavailable and/or is not signaled by the relevant access network, thereliability metric can be estimated at the traffic distribution entitybased, for example, on the subflow's quality, RTT, bandwidth, and/orother performance metrics.

In some example implementations, the traffic distribution entity selectsand may dynamically adjust the traffic distribution mode used inconnection with a given user equipment device in the next relevantperiod. Depending on the particulars of the network environment in whicha given implementation is provide, the period length may vary from a fewmilliseconds, to hundreds of milliseconds, or up to times measured onthe order of full seconds. As noted above with respect to FIGS. 5 and 7,a wide variety of different transmission modes. In some exampleimplementations, the traffic distribution modes may comprise at leastthe following, which are listed from most resource efficient to leastefficient:

(1) “Single-interface-only”: In this mode, only the primary network isused during the next period for transmission of all packets of a certainrelevant application, IP flow, and/or QoS flow.

(2) “Duplicate failed packets on the same/other interface”: In thismode, fast duplication over the other or same interface of packets basedon the fast detection that some TCP segments may be failing to meet therelevant latency and/or reliability requirements (which may be due, forexample, to lower layer failures).

(3) “Bi-casting serially”: In this mode, n replicas of each packet aretransmitted over one network, where n>=2.

(4) “Bi-casting in parallel”: In this mode, n replicas of each packetare transmitted over different networks, where n>=2.

It will be appreciated that in some example implementations, the trafficdistribution mode switch is triggered by the prediction that thereliability target may not be met with a certain probability or likewiseit may not be met by a certain offset (for example with an additional Xms of additional latency introduced).

In example implementations, each access network determines the actual orpredicted reliability metric that expresses the reliability which isbeing provided and/or can be provided in the next period (if available).The reliability metric could be expressed in terms of latency and/or RTTto deliver the current buffered traffic. In some such implementations,the reliability metric may also consider the relevant user equipmentdevice's SINR, RSRP, RSRQ, RSSI, cell load, status of L1/L2 ACK/NACK(which may involve HARQ processes in use and L1 ACK/NACK status, MAC/RLCACK/NACK status or the like, for example), and/or other factors.Granularity of the metric could be at a user, application, packet,sequence of packets, and/or priority/flagged packets level. In the caseof priority and/or flagged packets, the flagging could be, for example,placed in one or more fields in the protocol header (for example, ECN,IP header), or piggybacked with a relevant payload.

Referring now to FIG. 9, the operations performed by the apparatus 200of FIG. 2 in accordance with an example embodiment of the presentinvention are depicted as an example process flow 900. In this regard,the apparatus includes means, such as the processor 202, the memory 204,the user interface 206, the communication interface 208 or the like, forreceiving, at a traffic distribution entity in a hybrid networkenvironment, a first reliability probability indicator associated with afirst access network and a second reliability probability indicatorassociated with a second access network; determining, based at least inpart on the first reliability probability indicator and the secondreliability probability indicator, a combined reliability probabilityfor each traffic distribution mode within a plurality of trafficdistribution modes; ranking each traffic distribution mode with acombined reliability probability above a target reliability probability;and selecting a traffic distribution mode with a combined reliabilityprobability above the target reliability probability.

The apparatus includes means, such as the processor 202, the memory 204,the user interface 206, the communication interface 208 or the like, forreceiving, at a traffic distribution entity in a hybrid networkenvironment, a first reliability probability indicator associated with afirst access network and a second reliability probability indicatorassociated with a second access network. With reference to FIG. 9,process flow 900 may commence at block 902, which includes receiving afirst probability indicator associated with a first access network and asecond reliability probability indicator associated with a second accessnetwork. As discussed throughout this disclosure, many exampleimplementation of embodiments of the invention are directed to providingfor the efficient distribution of traffic in a hybrid network in amanner that meets the potentially mission critical reliability criteriaassociated with a user equipment device, such as a cyberphysical system(which may include, for example, an autonomous vehicle, drone, or pieceof industrial equipment). In some example implementations, of process900 in general and block 902 in particular, the first reliabilityprobability indicator is an indicator of a composite metric of a packetloss ratio and a latency associated with the first access network andwherein the second reliability probability indicator is an indicator ofa composite metric of a packet loss ratio and a latency associated withthe second access network. Moreover, in some example implementations,the first reliability indicator and the second reliability indicator areassociated a cyberphysical system operating within the hybrid network.Any approach to determining and/or conveying a reliability probabilitydescribed and/or otherwise contemplated herein may be used in connectionwith example implementations of block 902. Moreover, in some exampleimplementations of block 902 the hybrid network environment is amulti-path transmission control protocol (MPTCP) environment. However,it will be appreciated that example implementations of process 900and/or other embodiments of the invention may be realized in otherhybrid network environments.

It will also be appreciated that the reliability metrics used inconnection with example implementations of embodiments of the inventionmay be determined on a per service, application, device, user equipment,bearer, and/or flow basis. In some example implementations, it may beparticularly advantageous to define and/or use reliability probabilitiesor other related metrics that are associated with a particular piece ofuser equipment and/or other device within a network environment.

The apparatus also includes means, such as the processor 202, the memory204, the user interface 206, the communication interface 208 or thelike, for determining, based at least in part on the first reliabilityprobability indicator and the second reliability probability indicator,a combined reliability probability for each traffic distribution modewithin a plurality of traffic distribution modes. With reference to FIG.9, process flow 900 may continue to block 904, which includes computinga combined reliability probability for each traffic distribution modewithin a plurality of traffic distribution modes. As noted herein,particularly with reference to FIGS. 5 and 7 and the discussion relatedthereto, some example implementations of embodiments of the inventionare aimed towards identifying traffic distribution modes, such as singletransmission modes, parallel bicasting modes, serial bicasting modes,and/or combinations thereof which are capable of meeting reliabilityrequirements that may be necessary to ensure the proper operation ofcyberphysical systems and/or other network entities.

Moreover, in some example implementations, the first reliabilityindicator and the second reliability indicator are associated with atleast part of a cyberphysical system or a high-reliability low-latencycommunication system operating within the hybrid network. In suchimplementations, the combined reliability probability for each trafficdistribution mode within the plurality of traffic distribution modeswill also typically be associated with at least part of thecyberphysical system or the high-reliability low-latency communicationsystem operating within the hybrid network, such that the componentreliability probabilities and the combined reliability probability isdevice-specific or based at least in part on the characteristics of aparticular device. In some such example implementations, the targetreliability probability comprises a real-time reliability requirementassociated with the cyberphysical system or the high-reliabilitylow-latency communication system operating within the hybrid network. Asnoted herein, some example implementations (such as those involvingautonomous vehicles or drones, for example) involve potentiallymission-critical operations that require communications to be reliablyachieved within critical timeframes. Any of the approaches todetermining a combined reliability probability for a given trafficdistribution mode described and/or otherwise contemplated herein may beused in connection with example implementations of process flow 900.

The apparatus also includes means, such as the processor 202, the memory204, the user interface 206, the communication interface 208 or thelike, for ranking each traffic distribution mode with a combinedreliability probability above a target reliability probability. Withreference to FIG. 9, process flow 900 may proceed to block 906, whichincludes ranking each traffic distribution mode with a combinedreliability probability above a target reliability probability. Asdiscussed herein, a cyberphysical system, a user equipment device,and/or applications or other processes associated therewith may requirecommunications to be at or above a predetermined target reliabilityprobability threshold to ensure the safe and/or otherwise properoperation of the network given network entity and/or process. Moreover,since network resources may be limited, it may be necessary in somesituations to rank any acceptable traffic distribution mode to identifythe mode that best meets the requirements of the user equipment deviceand the hybrid network in which the device seeks to operate. In somesuch example implementations, ranking each traffic distribution modewith a combined reliability probability above a target reliabilityprobability comprises applying a predefined cost function to eachtraffic distribution mode. For example the cost function may allow forthe evaluation, weighting, and/or other ranking of traffic distributionmodes based on the network resources and/or other costs associated withimplementing the particular traffic distribution mode. In some exampleimplementations, other metrics may be involved in the ranking of one ormore traffic distribution modes. For example, metrics and/or otherinformation related to energy efficiency, the bandwidth required toaccomplish a transmission, and/or other factors may be used as rankingcriteria and/or incorporated into a cost function and/or other rankingrule set.

The apparatus also includes means, such as the processor 202, the memory204, the user interface 206, the communication interface 208 or thelike, for selecting a traffic distribution mode with a combinedreliability probability above the target reliability probability. Withreference to FIG. 9, process flow 900 may proceed to block 908, whichincludes selecting a traffic distribution mode with a combinedreliability probability above the target reliability probability. Any ofthe approaches described and/or otherwise contemplated herein may beused in connection with selecting a given traffic distribution mode. Insome example implementations, the selected mode will be the highestranked mode. In some example implementations, a traffic distributionentity may select and/or switch a traffic distribution mode based onreceipt of information associated with one or more reliabilityprobabilities.

With reference to FIGS. 5, 7 and 8, and the discussions related thereto,it will be appreciated that in some example implementations selecting atraffic distribution mode comprises causing a transmission to bedirected to the cyberphysical system via one subflow associated with thefirst access network. In some such example implementations, and in otherexample implementations, selecting a traffic distribution mode comprisescausing a bicasted transmission to be directed to the cyberphysicalsystem via one subflow associated with the first access network and viaone subflow associated with the second access network. Likewise, in someexample implementations of block 908, selecting a traffic distributionmode comprises causing a transmission to be directed to thecyberphysical system via one subflow associated with the first accessnetwork and causing a bicasted version of the transmission to bedirected to the cyberphysical system via the second access network. Itwill be appreciated that other traffic distribution modes may beselected, based on the particulars of the network environment in which agiven embodiment is implemented.

It will be appreciated that some implementations of embodiments of theinvention described and/or otherwise disclosed herein allow for users ofthe invention and/or other entities associated with the operation ofnetwork devices to ensure that the required level of reliability isprovided for cyber physical systems within hybrid networks. Inparticular, some example environments allow for the use and leveragingof network resources (including but not limited to disparate and/orlegacy network resources) to achieve the necessary reliability levelsrequired for certain application while using the network and/or userequipment resources in an efficient manner. Moreover, in implementationsthat allow for the rapid switching of traffic distribution modes and/orthe fast detection of potential changes in reliability probabilities,the ability to react upon situations where violation of the requirementscan be anticipated with certain likelihood may be advantageous. Forexample, in implementations involving autonomous driving or dronecontrol, example implementations of embodiments of the invention can beapplied and be advantageous in ensuring the necessary communicationreliability needed to safely and accurately operate such systems.

It will be appreciated that, in some situations, none of the combinedreliability probabilities of the traffic distribution modes will satisfythe relevant target reliability probability. For example, in a givensituation, the target reliability probability may be set at a level thatis high enough that the available access networks or other networkpathways within a hybrid network environment may not be able to becombined in a manner that meets the threshold. In other situations,hybrid network may be damaged, incompletely implemented, and/orotherwise subject conditions that limit the reliability of the variousrelevant network components. In some such example implementations, thecombined reliability probabilities of the available traffic distributionmodes may be ranked, and the highest ranked traffic distribution modemay be selected. In some such example implementations, the apparatusand/or other relevant network element may check and update theindividual reliability probabilities and/or combined reliabilityprobabilities at a more frequent rate, with the goal of identifyingchanges in the reliability probabilities that may allow for theselection of a traffic distribution mode that meets the targetrequirements. In some example implementations, selecting a trafficdistribution mode that does not satisfy the requirements of the targetdistribution mode may also involve the initiation and/or coordination ofactions designed to limit risks associated with the relevant userdevice. For example, in the case of cyberphysical systems such as anautonomous car, train, or drone, the selection of a traffic distributionmode with a reliability probability that does not satisfy the relevanttarget requirements may trigger the transmission of commands that wouldstop the car, slow down the train and/or land the drone, as appropriate.

As described above, FIG. 9 illustrates a flowchart of an apparatus 200,method, and computer program product according to example embodiments ofthe invention. It will be understood that each block of the flowchart,and combinations of blocks in the flowchart, may be implemented byvarious means, such as hardware, firmware, processor, circuitry, and/orother devices associated with execution of software including one ormore computer program instructions. For example, one or more of theprocedures described above may be embodied by computer programinstructions. In this regard, the computer program instructions whichembody the procedures described above may be stored by the memory device204 of an apparatus employing an embodiment of the present invention andexecuted by the processor 202 of the apparatus. As will be appreciated,any such computer program instructions may be loaded onto a computer orother programmable apparatus (such as hardware, for example) to producea machine, such that the resulting computer or other programmableapparatus implements the functions specified in the flowchart blocks.These computer program instructions may also be stored in acomputer-readable memory that may direct a computer or otherprogrammable apparatus to function in a particular manner, such that theinstructions stored in the computer-readable memory produce an articleof manufacture the execution of which implements the function specifiedin the flowchart blocks. The computer program instructions may also beloaded onto a computer or other programmable apparatus to cause a seriesof operations to be performed on the computer or other programmableapparatus to produce a computer-implemented process such that theinstructions which execute on the computer or other programmableapparatus provide operations for implementing the functions specified inthe flowchart blocks.

Accordingly, blocks of the flowchart support combinations of means forperforming the specified functions and combinations of operations forperforming the specified functions. It will also be understood that oneor more blocks of the flowchart, and combinations of blocks in theflowchart, can be implemented by special purpose hardware-based computersystems which perform the specified functions, or combinations ofspecial purpose hardware and computer instructions.

In some embodiments, certain ones of the operations above may bemodified or further amplified. Furthermore, in some embodiments,additional optional operations may be included. Modifications,additions, or amplifications to the operations above may be performed inany order and in any combination.

Many modifications and other embodiments of the inventions set forthherein will come to mind to one skilled in the art to which theseinventions pertain having the benefit of the teachings presented in theforegoing descriptions and the associated drawings. Therefore, it is tobe understood that the inventions are not to be limited to the specificembodiments disclosed and that modifications and other embodiments areintended to be included within the scope of the appended claims.Moreover, although the foregoing descriptions and the associateddrawings describe example embodiments in the context of certain examplecombinations of elements and/or functions, it should be appreciated thatdifferent combinations of elements and/or functions may be provided byalternative embodiments without departing from the scope of the appendedclaims. In this regard, for example, different combinations of elementsand/or functions than those explicitly described above are alsocontemplated as may be set forth in some of the appended claims.Although specific terms are employed herein, they are used in a genericand descriptive sense only and not for purposes of limitation.

1-28. (canceled)
 29. A method comprising: receiving, at a trafficdistribution entity in a hybrid network environment, a first reliabilityprobability indicator associated with a first access network and asecond reliability probability indicator associated with a second accessnetwork; determining, based at least in part on the first reliabilityprobability indicator and the second reliability probability indicator,a combined reliability probability for each traffic distribution modewithin a plurality of traffic distribution modes; ranking each trafficdistribution mode with a combined reliability probability above a targetreliability probability; and selecting a traffic distribution mode witha combined reliability probability above the target reliabilityprobability.
 30. The method of claim 29, wherein the first reliabilityprobability indicator is an indicator of a composite metric of a packetloss ratio and a latency associated with the first access network andwherein the second reliability probability indicator is an indicator ofa composite metric of a packet loss ratio and a latency associated withthe second access network.
 31. The method of claim 29, wherein thehybrid network environment is a multi-path transmission control protocol(MPTCP) environment.
 32. The method of claim 29, wherein the firstreliability indicator and the second reliability indicator areassociated with at least part of a cyberphysical system or ahigh-reliability low-latency communication system operating within thehybrid network; wherein the combined reliability probability for eachtraffic distribution mode within the plurality of traffic distributionmodes is associated with at least part of the cyberphysical system orthe high-reliability low-latency communication system operating withinthe hybrid network; and wherein the target reliability probabilitycomprises a real-time reliability requirement associated with thecyberphysical system or the high-reliability low-latency communicationsystem operating within the hybrid network.
 33. The method of claim 29,wherein ranking each traffic distribution mode with a combinedreliability probability above a target reliability probability comprisesapplying a predefined cost function to each traffic distribution mode.34. The method of claim 32, wherein selecting the traffic distributionmode with a combined reliability probability above the targetreliability probability comprises causing a transmission to be directedto the cyberphysical system via one subflow associated with either thefirst access network or the second access network.
 35. The method ofclaim 32, wherein selecting the traffic distribution mode with acombined reliability probability above the target reliabilityprobability comprises causing a bicasted transmission to be directed tothe cyberphysical system via one subflow associated with the firstaccess network and via one subflow associated with the second accessnetwork.
 36. The method of claim 32, wherein selecting the trafficdistribution mode with a combined reliability probability above thetarget reliability probability comprises causing a transmission to bedirected to the cyberphysical system via one subflow associated with thefirst access network and causing a bicasted version of the transmissionto be directed to the cyberphysical system via the second accessnetwork.
 37. The method of claim 29, wherein selecting a trafficdistribution mode with a combined reliability probability above thetarget reliability probability comprises selecting the trafficdistribution mode with the highest ranking.
 38. An apparatus comprisingat least one processor and at least one memory storing computer programcode, the at least one memory and the computer program code configuredto, with the processor, cause the apparatus to at least: receive, in ahybrid network environment, a first reliability probability indicatorassociated with a first access network and a second reliabilityprobability indicator associated with a second access network;determine, based at least in part on the first reliability probabilityindicator and the second reliability probability indicator, a combinedreliability probability for each traffic distribution mode within aplurality of traffic distribution modes; rank each traffic distributionmode with a combined reliability probability above a target reliabilityprobability; and select a traffic distribution mode with a combinedreliability probability above the target reliability probability. 39.The apparatus of claim 38, wherein the first reliability probabilityindicator is an indicator of a composite metric of a packet loss ratioand a latency associated with the first access network and wherein thesecond reliability probability indicator is an indicator of a compositemetric of a packet loss ratio and a latency associated with the secondaccess network.
 40. The apparatus of claim 38, wherein the hybridnetwork environment is a multi-path transmission control protocol(MPTCP) environment.
 41. The apparatus of claim 38, wherein the firstreliability indicator and the second reliability indicator areassociated with at least part of a cyberphysical system or ahigh-reliability low-latency communication system operating within thehybrid network; wherein the combined reliability probability for eachtraffic distribution mode within the plurality of traffic distributionmodes is associated with at least part of the cyberphysical system orthe high-reliability low-latency communication system operating withinthe hybrid network; and wherein the target reliability probabilitycomprises a real-time reliability requirement associated with thecyberphysical system or the high-reliability low-latency communicationsystem operating within the hybrid network.
 42. The apparatus of claim38, wherein ranking each traffic distribution mode with a combinedreliability probability above a target reliability probability comprisesapplying a predefined cost function to each traffic distribution mode.43. The apparatus of claim 41, wherein selecting the trafficdistribution mode with a combined reliability probability above thetarget reliability probability comprises causing a transmission to bedirected to the cyberphysical system via one subflow associated witheither the first access network or the second access network.
 44. Theapparatus of claim 41, wherein selecting the traffic distribution modewith a combined reliability probability above the target reliabilityprobability comprises causing a bicasted transmission to be directed tothe cyberphysical system via one subflow associated with the firstaccess network and via one subflow associated with the second accessnetwork.
 45. The apparatus of claim 41, wherein selecting the trafficdistribution mode with a combined reliability probability above thetarget reliability probability comprises causing a transmission to bedirected to the cyberphysical system via one subflow associated with thefirst access network and causing a bicasted version of the transmissionto be directed to the cyberphysical system via the second accessnetwork.
 46. The apparatus of claim 38, wherein selecting a trafficdistribution mode with a combined reliability probability above thetarget reliability probability comprises selecting the trafficdistribution mode with the highest ranking.
 47. A computer programproduct comprising at least one non-transitory computer-readable storagemedium having computer-executable program code instruction storedtherein, the computer-executable program code instructions comprisingprogram code instructions configured to: receive, in a hybrid networkenvironment, a first reliability probability indicator associated with afirst access network and a second reliability probability indicatorassociated with a second access network; determine, based at least inpart on the first reliability probability indicator and the secondreliability probability indicator, a combined reliability probabilityfor each traffic distribution mode within a plurality of trafficdistribution modes; rank each traffic distribution mode with a combinedreliability probability above a target reliability probability; andselect a traffic distribution mode with a combined reliabilityprobability above the target reliability probability.
 48. The computerprogram product of claim 47, wherein the first reliability probabilityindicator is an indicator of a composite metric of a packet loss ratioand a latency associated with the first access network and wherein thesecond reliability probability indicator is an indicator of a compositemetric of a packet loss ratio and a latency associated with the secondaccess network.